Item Classification System, Device and Method Therefor

ABSTRACT

An item classification system for use in an item handling system is disclosed. The classification system comprises processing means configured to: process an image of an item to determine, based on a first model ( 101 ), one or more predetermined first item types, each first item type defined by one or more first item characteristics; process the image to determine, based on the first model ( 101 ), a first probability associated with each first item type wherein each first probability is indicative of the likelihood that the item has the first characteristics defining each determined first item type; process the image to determine, based on a second model ( 103 ), one or more predetermined second item types, each second item type defined by one or more second item characteristics; process the image to determine, based on the second model ( 103 ), a second probability associated with each second item type wherein each second probability is indicative of the likelihood that the item has the second characteristics defining each second item type; and classify the item according to each first item type and each second item type and the probability associated with each first item type and the probability associated with each second item type.

FIELD OF THE INVENTION

This invention relates to item classification or recognition methods andsystems. Further, this invention relates to image processing methods andsystem. It is particularly, but not exclusively, concerned with baggageclassification and handling methods and systems, for example operatingat airports, seaports, train stations, other transportation hubs ortravel termini.

BACKGROUND OF THE INVENTION

Baggage performance has become very high priority in the majority ofairlines. The Air Transport Industry transports some 2.25 billion bagsannually. While 98% of all bags reach their destination at the same timeas the owner, the 2% mishandled bags have been receiving increasinglynegative press coverage and passenger complaints are on the rise.

Bags that are mishandled, particularly if they have lost their tags, areoften very difficult to identify. This is because a passenger'sdescription of their bag is subjective, and therefore matching aparticular description to a bag is very difficult indeed and sometimesimpossible. This is particularly the case in the aviation industry wherea very large number of bags are transported annually. This issue iscompounded by potential language difficulties.

Although there exists a standardized list of IATA™ bag categories, thecurrent baggage mishandling process suffers from a number of problems:

-   -   It is a labour-intensive labelling process by examining each bag    -   If a bag is not clearly one colour or another, the labelling may        well not be consistent    -   Both human error and disagreement may impact how a bag is        labelled and recorded    -   Staff must be trained in understanding the baggage categories

Further, conventional colour determination algorithms work based on adistance function which determines a distance between two points A, B ina 3-d colour space such as HSV or RGB. According to this scheme, unknowncolours are categorised according whether known colours defined by thepoints A or B in a 3D colour space are closest to the unknown colour inthe 3D colour space.

A problem with this approach is that the closest colour in terms of thedistance function is not necessarily the correct predetermined colour.This due to fact that human colour perception varies betweenindividuals. For example, one person may categorise an orange colour asyellow.

To solve this problem, a mapping scheme is performed which categorisesbag colours according to predetermined colour types. The colour of a bagis determined by mapping a bag colour to one of a plurality of differentpredetermined colours or labels according to a colour definition table.

SUMMARY OF THE INVENTION

Embodiments of the invention seek to address these problems by usingartificial intelligence to classify or/and identify a bag or bags basedon a single image associated with a bag. This may be performed atcheck-in, or subsequent to check-in using a computer or server or mobiletelephone or other portable computing device. The image is thenprocessed, noting what the probable bag properties are, according to theclassification, usually with an associated degree of certainty. This maybe performed for each bag processed at an airport. These categories maythen be recorded in a database and processed using a baggagereconciliation program, or displayed, to assist in baggage recovery.

According to some aspects, an item classification system for use in anitem handling system. The classification system comprises a processingmeans configured to process an image of an item to determine, based on afirst model, one or more predetermined first item types, each first itemtype defined by one or more first item characteristics, process theimage to determine, based on a second model (103), one or morepredetermined second item types, each second item type defined by one ormore second item characteristics; and classify the item according toeach first item type and each second item type. Preferably, each item isfurther classified according to the probability associated with eachfirst item type and the probability associated with each second itemtype.

The invention is defined in the appended claims to which referenceshould now be made.

One advantage of being able to classify an item using only a singleimage is because existing systems do not need to be modified to capturemultiple images. Therefore, embodiments of the invention avoidadditional infrastructure such as multiple cameras being needed to takedifferent views of an item being classified.

Currently, there are no existing automated methods for performing such aprocedure.

Accordingly, embodiments of the invention may using a one or morecomputer vision models to identify various different aspects of an itemor bag, such as type, material, colour, or external element properties.Preferably three models are used. This has approximately a 2%performance improvement compared to using one or two models.

Even more preferably the outputs or classifications from 3 models arecombined. This has approximately a 15% performance improvement comparedto not combining the outputs of 3 models. Preferably, embodiments of theinvention may comprise resizing of an input image to improve performancespeed.

Embodiments of the invention use machine learning techniques which use adataset of bag images to generate classifications for each image basedon one or more categories. In addition, embodiments of the invention maycomprise processing using a rules engine, image manipulation techniques,such as white-balancing.

Embodiments of the invention may generate estimated probabilities foreach of the possible bag types which a bag belongs to, including theprobability that a bag has a label and location of the labels as well asother groups of characteristics of the bag. These may be stored in adatabase.

Compared to existing item recognitions systems, embodiments of theinvention have the advantage that:

-   -   Automation results in a faster process than a person examining        each bag by hand;    -   The marginal cost to analyse each bag is be lower for a        computerized system;    -   The images are stored to iterate on the process, resulting in        accuracy improvements over time;    -   It is easier to integrate embodiments of the invention with        electronic systems; and    -   Embodiments of the invention have the advantage that they        provide objective, rather than subjective, assessments of        colour.

According to a further aspect of the present invention, a colourdetermination process is disclosed. The colour determination process maybe performed alternatively or in addition to item classification or/andidentification.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention will now be described, by way of exampleonly, and with reference to the accompanying drawings, in which:

FIG. 1 is a schematic diagram showing the main functional componentsaccording to an embodiment of the invention;

FIG. 2 is an exemplary image of training data used to train the neuralnetwork;

FIG. 3 is an exemplary image of a passenger's bag captured by a cameralocated at a bag drop desk;

FIG. 4 is a flow diagram showing the main steps performed by anembodiment of the invention;

FIG. 5 is an exemplary further image of a passenger's bag captured by acamera located at a bag drop desk;

FIG. 6 is a schematic diagram showing the main functional componentsaccording to a further embodiment;

FIG. 7 is a flow diagram showing the main steps performed by a furtherembodiment of the invention;

FIG. 8 shows a colour tree diagram according to an embodiment of theinvention; and

FIG. 9 shows an exemplary further image of a passenger's bag captured bya camera located at a bag drop desk which is displayed using a display,together with the determined item types, characteristics and associatedprobabilities.

DETAILED DESCRIPTION

The following exemplary description is based on a system, apparatus, andmethod for use in the aviation industry. However, it will be appreciatedthat the invention may find application outside the aviation industry,particularly in other transportation industries, or delivery industrieswhere items are transported between locations.

The following embodiments described may be implemented using a Pythonprogramming language using for example an OpenCV library. However, thisis exemplary and other programming languages known to the skilled personmay be used such as JAVA.

System Operation

An embodiment of the invention will now be described referring to thefunctional component diagram of FIG. 1, also referring to FIGS. 2 and 3as well as the flow chart of FIG. 4.

Usually, the messaging or communication between different functionalcomponents is performed using the XML data format and programinglanguage. However, this is exemplary, and other programming languages ordata formats may be used, such as REST\JSON API calls. These may becommunicated over HTTPS using wired or wireless communications protocolswhich will be known to the skilled person. JSON calls may also beadvantageously used.

Usually, the different functional components may communicate with eachother, using wired or wireless communication protocols which will beknown to the skilled person. The protocols may transmit service calls,and hence data or information between these components. Data within thecalls is usually in the form of an alpha-numeric string which iscommunicated using wired or wireless communication protocols.

The system may comprise any one or more of 5 different models. Each ofthe models may run on a separate computer processor or server, althoughit will be appreciated that embodiments of the invention may inprinciple run on a single computer or server. Usually, a wired orwireless communications network is used. This may communicatively coupleone or more of the functional components shown in FIG. 1 together toallow data exchange between the component(s). It may also be used toreceive an image of a bag captured by a camera or other recording means109. Usually, the camera or recording means is positioned on or within abag drop kiosk or desk, or a self-service bag drop machine at anairport. It will be appreciated that the image comprises sample valuesor pixels.

It will be also appreciated that many such cameras or recording meansmay be coupled to a central computer or server which classifies eachbag, as will be described in further detail below.

In all cases, wired or wireless communications protocols may be used toexchange information between each of the functional components.

The computer or server comprises a neural network. Such neural networksare well known to the skilled person and comprise a plurality ofinterconnected nodes. This may be provided a web-service cloud server.Usually, the nodes are arranged in a plurality of layers L1, L2, . . .LN which form a backbone neural network. For more specialised imageclassification, a plurality of further layers are coupled to thebackbone neural network and these layers may perform classification ofan item or regression such as the function of determining a bounding boxwhich defines a region or area within an image which encloses the itemor bag.

As shown in FIG. 1 of the drawings one or more models 101, 103, 105,107, 109 may be used to classify or/and identify an item of baggage.

Each model may be trained using a convolutional neural network with aplurality of nodes.

Each node has an associated weight. The neural network usually has oneor more nodes forming an input layer and one or more nodes forming anoutput layer. Accordingly, the model may be defined by the neuralnetwork architecture with parameters defined by the weights.

Thus, it will be appreciated that neural network is usually trained.However, training of neural networks is well known to the skilledperson, and therefore will not be described in further detail.

Nevertheless, the inventor has found that a training data set of 9335images of bags with a test size of 4001 was found to provide acceptableresults when the trained neural network was used to classify bag types.The test size is the number of images used to validate the result. Onespecific example of a neural network is the RetinaNet network neuralnetwork having 50 layers (ResNet50) forming a backbone neural network,although more or less layers may be used and it will be appreciated thatother backbone neural networks may be used instead of the ResNet50neural network. RetinaNet is an implementation of loss function formanually tuned neural network architectures for the object detection andsegmentation, and will be known to the skilled person. Thus, each ofmodels 101, 103, 105, 107, 109 may be implemented using RetinaNet.

The following machine learning algorithms may also be used to implementembodiments of the invention. This shows accuracy metrics of differentmachine learning algorithms.

Machine Learning Algorithm Accurancy LightGBM 0.863 Random Forest 0.858K-Nearest Neighbours 0.798 SVM Linear Kernel 0.861 SVM Polynomial Kernel0.861

Usually, the neural network is remotely accessed by wired or wirelesscommunication protocols which will be known to the skilled person.

Each image in the training data set has an associated type definitionand/or material definition and a bounding box defining the location ofthe bag within the image was defined. Thus, it will be appreciated thatthe bounding box may represent a possible region of interest in theimage. Usually, the bounding box is defined by the coordinates of arectangular border that encloses all or a portion of an image. Anexternal element definition was also associated with each image.

The type model 101 is trained to classify a bag according to a number ofpredetermined categories. The model 101 is trained using the trainingdata set of images to determine a bag type. Separate model 103 istrained using the training data set of images to determinecharacteristics of the bag external elements. Material model 105 istrained using the training data set of images to determine a materialtype of the bag. An exemplary image included in the training data set isshown in FIG. 2 of the drawings. The training data comprises an image ofa bag and associated CSV values defining x and y coordinates of thebounding box. The CSV values associated with the image of FIG. 2 areshown in table 1.

TABLE 1 The bounding boxes of each image are defined by the bottom leftx coordinate (Blx), the bottom right y coordinate (Bly), the top right x(Trx) coordinate and the top right y (Try) coordinate. Each bounding boxhas an associated label and image file name. Bounding boxes areexplained in further detail below. Coordinates File name Blx Bly Trx TryLabel /mnt/dump/BagImages/images/1004.png 154 38 364 412 D/mnt/dump/BagImages/images/1004.png 154 38 364 412 T22D/mnt/dump/BagImages/images/1004.png 154 38 364 412 BK/mnt/dump/BagImages/images/1004.png 307 218 360 259 wheel/mnt/dump/BagImages/images/1004.png 294 361 337 401 wheel/mnt/dump/BagImages/images/1004.png 225 57 248 89 combo_lock/mnt/dump/BagImages/images/1004.png 196 57 225 112 zip_chain/mnt/dump/BagImages/images/1004.png 183 140 210 381 zip_chain

Once one or more of the models have been trained using the trainingdata, embodiments of the invention use one or more of the trained modelsto detect for material, type and external elements of the bag. The typemodel 101 categorises an image of a bag according to one or more of thefollowing predetermined categories shown in table 2:

TABLE 2 Type Precisions of different baggage classifications determinedaccording to an embodiment of the invention. Label Name Precision N T01Horizontal design Hard Shell 0.000 6 T02 Upright design 0.889 476 T03Horizontal design suitcase Non-expandable 0.000 3 T05 Horizontal designsuitcase Expandable 0.000 5 T09 Plastic/Laundry Bag 0.000 3 T10 Box0.939 33 T12 Storage Container 0.000 5 T20 Garment Bag/Suit Carrier0.000 5 T22 Upright design, soft material 0.000 26 T22D Upright design,combined hard and soft 0.944 748 material T22R Upright design, hardmaterial 0.932 2062 T25 Duffel/Sport Bag 0.379 29 T26 Lap Top/OvernightBag 0.357 42 T27 Expandable upright 0.397 267 T28 Matted woven bag 0.0002 T29 Backpack/Rucksack 0.083 12

In addition to the types identified in Table 2, the following additionalbag categories may be defined. A label of Type 23 indicates that the bagis a horizontal design suitcase. A label of Type 6 indicates that thebag is a brief case. A label of Type 7 indicates that the bag is adocument case. A label of Type 8 indicates that the bag is a militarystyle bag. However, currently, there are no bag types indicated by thelabels Type 4, Type 11, Type 13-19, Type 21, or Type 24.

In Table 2, N defines the number of predictions for each bag category orname, for example “Upright design”, and the label is a standardlabelling convention used in the aviation industry. Preferably, afiltering process may be used to remove very dark images based on anaverage brightness of pixels associated with the image.

The external elements model 103 categorises an image of a bag accordingto one or more of the following predetermined categories shown in table3:

TABLE 3 Different external elements classifications and precisions withscore threshold = 0.2. If the prediction gives a probability of lessthan 0.2, then the data is not included. The buckle and zipcategorisations may advantageously provide for improved itemclassification, which will be explained in further detail below. NameRecall N_act Precision N_pred buckle 0.300 40 0.203 59 combo_lock 0.9211004 0.814 1137 retractable_handle 0.943 421 0.827 480 straps_to_close0.650 197 0.621 206 wheel 0.988 1549 0.932 1642 zip 0.910 1539 0.9141531

The material model 105 categorises an image of a bag according to one ormore of the following predetermined categories shown in table 4:

TABLE 4 Different material classifications and precisions. label nameprecision N D Dual Soft/Hard 0.816 437 L Leather 0.000 3 M Metal 0.000 3R Rigid (Hard) 0.932 1442 T Tweed 0.444 57

Baggage Classification on Bag Drop

An exemplary bag classification process will now be described withreference to FIGS. 1, 3, 4, and 5 of the drawings.

A passenger arrives at a bag drop kiosk and deposits their bag on thebelt 301, 501. The camera 109 or belt 301 or 501 or both detect, at step401, a that a bag 303, 503 has been placed on the belt. This may beperformed using image processing techniques by detecting changes insuccessive images in a sequence of images or by providing a weightsensor coupled to the belt. An image or picture of the bag 300, 500 isthen taken using the camera 109 in response to detection of the bag.

In either case, the piece of baggage is detected, and this may be usedas a trigger to start the bag classification process. Alternatively, theimage may be stored in a database and the classification may beperformed after image capture and storage.

At step 403, one or more of the models 101, 103, 105, process the image300, 500 of the bag captured by the image capture means or camera 109.In principle, the models may operate in series or parallel, but parallelprocessing is preferable.

The results of the processing of the image 300, 500 are shown in tables4, 5 and 6.

TABLE 4 An exemplary RetinaNet Type model output. In this example, themodel 101 has determined that the bag may belong to three possibledifferent categories of bags: T02 is an upright design with no zip, T22Ris an upright design with hard material with a zip and T27 is anexpandable upright design. >>>models.type_model.image_predict(img).iloc[:,:2] score label 0 0.604188T02 0 0.471872 T22R 0 0.222365 T27

Under certain circumstances, model 101 may correctly classify a bag asthe bag type having an associated score or probability which is thehighest, depending upon the image and the position which the bag isplaced on the belt.

However, because of poor image quality or the position in which a bag isplaced on the belt, the model 101 may not correctly classify the bag.

Therefore, it is advantageous that the external elements model 103 alsooperates on the image 300 or 500. This may be done sequentially or inparallel to the processing performed by model 101.

TABLE 5 An exemplary RetinaNet External model output. The combo lock andzip slider, and wheel external elements categorisations orclassifications appear more than once. This is because the model foundtwo of the features or characteristics. >>>models.external_model.image_predict(img).iloc[:,:2] score label 00.568917 combo_lock 0 0.447264 combo_lock 0 0.381501 zip_slider 00.359150 wheel 0 0.338619 retractable_handle 0 0.301219 combo_lock 00.255498 buckle 0 0.254586 zip_slider 0 0.216772 wheel

Under certain circumstances, the external elements model 103 may alsocorrectly classify a bag as the bag type having an associated score orprobability which is the highest, depending upon the image and theposition which the bag is placed on the belt.

However, because of poor image quality or the position in which a bag isplaced on the belt, model 103 may not correctly classify the bag.

Therefore, it is advantageous that the material model 105 also operateson the image 300 or 500. This may be done sequentially or in parallel tothe processing performed by models 101 and 103.

TABLE 6 An exemplary RetinaNet Material model output. >>>models.material_model.image_predict(img).iloc[:,:2] score label 00.983209 R

In the example shown in table 6, the model 105 has determined a singlerigid bag type.

As shown in FIG. 1 of the drawings, the outputs from each model 101,103, 105 shown in the specific examples above may be combined byweighting the classifications determined by model 101 based on theresults output from model 103 or/and 105.

TABLE 7 The LightGBM model 107 output. R means that the bag is rigid >>>models.predict(img) ( label score 0 T02 0.297951 1 T22R 0.475171 2 T270.212052

Thus, as will be seen from table 7, the type T22R has been more heavilyweighted, and therefore the bag is correctly classified in this exampleas type T22R, rather than the type T02 determined by model 101.

Thus, it will be appreciated that each of the models 101, 103, and 105are separately trained based on training data according to differentgroups of characteristics. This is in contrast to conventional neuralnetwork methods which learn multiple labels for a single image using asingle neural network. Thus, embodiments of the invention train using aplurality of different labels. The external elements model is separatelytrained based on the appreciation that this does not require knowledgeof other bag descriptions.

The light GBM model 107 may be trained using training data in order toweight the outputs from each of models 101, 103, 105 in order tocorrectly classify or categorise a bag.

Accordingly, it will be appreciated that embodiments of the inventionare able to correctly determine a predetermined bag type with anincreased probability.

This may have a 15% improvement in detection precision compared to usingknown image processing techniques such as conventional object detectionand classification methods which simply train a convolutional neuralnetwork model with labels and bounding boxes for the type. This is basedon the appreciation that some bag types are very similar. For example,the types for a hard-upright bag with a zip and a without a zip.Following the conventional method, in testing, most of themisclassifications have been from these similar types.

TABLE 8 Comparative performance analysis of the outputs of the Light GBMmodel 107 compared to the type model 101. Type precision Precision usingusing type LightGBM Label model model name 101 107 N T01 0 0 6 T020.761566 0.888655 476 T03 0 0 3 T05 0 0 5 T09 0 0 3 T10 0.5384620.939394 33 T12 0 0 5 T20 0 0 5 T22 0 0 26 T22D 0.701789 0.94385 748T22R 0.789194 0.93162 2062 T25 0.333333 0.37931 29 T26 0.136364 0.35714342 T27 0.355828 0.397004 267 T28 0 0 2 T29 0 0.083333 12

Table 8 shows the results of the retina net model 101 performing alone,as well as the results of the of combining and weighting the outputsfrom each of the models 101, 103, and 105 generated using light GBMmodel 107. It will be seen For example, using type model 101 on its own,for 476 images, there is approximately a 76% probability that the typemodel has correctly determined 476 bags as Type 02—upright design.However, when the light GBM precision model combines or/and weights theoutputs the possible bag types and associated probabilities from each ofmodel 101, 103, and 105, the probability that the 476 bags have beencorrectly classified as Type 02—upright design increases toapproximately 89%. This represents a significant performanceimprovement.

Thus, it will be appreciated that a model 107 may be trained usingLightGBM with outputs from the Type model 101, External Elements model103 and Material model 105 as categorical features. Having a descriptionof the Material and which External Elements there are provides importantcontext features which may improve accuracy.

The Material model 105 and Type RetinaNet model 101 may be trained usingtraining techniques which will be known to the skilled person. However,for the external elements model, embodiments of the invention caninclude new labels not defined by the industry, such as zip chain, zipslider and buckle. These may be used as additional features for trainingof the LightGBM model 107.

For example, with reference to FIG. 5 of the drawings, it will beappreciated that, possibly due to poor lighting, the RetinaNet 101 modelwas unable to determine correct class T22R Upright Design, Hard Material(with zip), and gives higher percentage to T02 Upright design (no zip).Using the RetinaNet External Elements model 103 which shows that zipsare likely to be present and allows the LightGBM model 107 to reducethis percentage and correctly classify a bag 109.

As shown in FIG. 9 the results of the classification or categorisationmay be output to a User Interface, at step 709. This may beadvantageously used by an agent for improved retrieval of a lost bag.

In addition, the item handling system and classification system may beconfigured in some implementations to process the item according to thedetermined item type. For example, bags which are identified as beingunsuitable for processing along one baggage processing channel may bediverted along an alternative path based on the categorisation. Forexample, items classified according to label T29 as a Backpack/Rucksackusually include one or more shoulder straps. Such shoulder straps caneasily become entangled in item processing equipment. Accordingly, itemsclassified according to label T29 may be processed by a first itemprocessing channel which is adapted to process such items.

Other items not categorised according to label T29 may be processed by asecond item processing channel which is not adapted to process suchitems. The item handling system may be configured to output a signal toa diverting or moving means such as pivotable arm including an actuatorwhich diverts items along the first processing channel or secondprocessing channel depending upon the determined item classification andthus the output signal.

Embodiments of the invention may be advantageously used to locatemissing or lost items.

This may be performed by searching a data base or storage means for anitem having characteristics corresponding to the determinedclassification of the item. Location data and data defining a time whenan item was detected may also be stored in the database and associatedwith each item.

Thus, a processor may be configured to search a database for itemshaving associated location data and data defining a time when the itemwas detected which is associated with the determined itemclassification.

Thus, it will be appreciated that when a bag or item is missing or lost,the processor may advantageously search a database for matching bagswith the characteristics during a predetermined time period atpredetermined location. This has the benefit that missing items may bemore quickly located.

Determining the Colour of a Bag

Alternatively or in addition to classifying a bag type as previouslydescribed, an embodiment of the invention will now be described whichclassifies a region of an image according to a predetermined colourclassification. Thus, it will be appreciated that the colourclassification process described below may be combined with thepreviously described bag or item classification, in order to be able tomore accurately identify or classify bags.

Colour classification may be performed by a colour mapping processaccording to a plurality of different colour definitions. These may beclassified according to the hue, saturation and value (H, S and V)definitions of a plurality of different colour categorisations in whichthe different colours are defined according to the values defined in theTable 9.

TABLE 9 The H, S, and V definitions of a number of predetermineddifferent colour classifications. Label Colour H V S wt white 0 100 0 bkblack 0 0 0 gy grey 0 35 0 gy grey 0 67 0 bu blue 203 41 77 bu blue 20688 55 bu blue 187 45 94 pu purple 270 53 50 pu purple 300 47 54 rd red 084 71 rd red 343 92 34 rd red 356 68 65 yw yellow 53 100 72 yw yellow 3096 73 yw yellow 40 99 68 be beige 58 100 28 be beige 36 91 35 bn brown44 37 43 bn brown 30 29 33 gn green 129 40 60 gn green 66 85 69 gn green88 67 72 bu1 blue 221 64 40 bu2 blue 220 33 39 bu3 blue 225 50 31

The values and labels bu1, bu2, and bu3 shown in bold are colourdefinitions which allow for a more precise colour determination of bluebags. The following describes how embodiments of the invention mayuniquely map a bag colour to a single one of the plurality of differentpredetermined colour classifications shown in Table 9.

Embodiments of the invention use certain rules for ranges of HSVassigned to colour instead of a distance function.

This will be described referring to the functional component diagram ofFIG. 6, the flow diagram of FIG. 7, and the colour tree diagram of FIG.8.

A passenger arrives at a bag drop kiosk and deposits their bag on thebelt 301, 501. The camera 109 or belt 301 or 501 or both detect, at step701, that a bag 303, 503 has been placed on the belt. This may beperformed using image processing techniques by detecting changes insuccessive images in a sequence of images or by providing a weightsensor coupled to the belt. An image or picture of the bag 300, 500 isthen taken using the camera 109, 609.

In either case, the piece of baggage is detected, and this may be usedas a trigger to start the colour classification process. Alternatively,the image may be stored in a database and the classification may beperformed after image capture and storage.

Each model may output a class label, a score or probability and abounding box for any object detected within the image input. As the type101, 601, and Material Models 105, 605 are trained using bounding boxesaround the whole bag, any prediction also outputs a predicted boundingbox for a bag, as shown in the specific example of Table 1.

Accordingly, the colour predict model may use the previously describedand trained RetinaNet models for Type 101, 609 and Material 105, 605 inorder to determine a bounding box around an item of baggage.

The colour prediction process may then use the bounding box with thehighest score as an input into a grab cut function which performsforeground/background selection to extract an accurate cut out of a bag.

Alternatively, the colour prediction process may separately determine abounding box for a bag within an image. This has the advantage that thecolour prediction process can be applied without necessarily determininga bag type using the type model 101 or the bag material using thematerial model 105.

In either case, a foreground portion of an image including the bag isextracted using the portion of the image within the bounding box. Thisportion of the image is input into the grab-cut function at step 703.The grab-cut function is a well-known image processing technique whichwill be known to the skilled person, but however other techniques may beused. The grab-cut function is available athttps://docs.opencv.orq/3.4.2/d8/d83/tutorial_py_grabcut.html

If the camera or image capture means generates an image according to anRGB colour space, then an average RGB colour value is then determinedfrom the portion of the image containing the bag or in other words theportion of the image output from the grab-cut function, at step 705.Alternatively, the image may be defined according to an HSV colourspace, in which case an average HSV colour value is determined from theportion of the image output from the grab-cut function, at step 705.

Optionally, the average colour determined at step 705 may be input intoa random forest model 619. The average colour may be determined using awell-known function such as a median function, a mode function or a meanfunction. Embodiments of the invention preferably used a mean functionto calculate the mean value of an array of elements. This may becalculated by summing the values of the array of elements and dividingthe sum by the number of elements in the array.

Irrespective of the type of colour space used, a single average over aplurality of channels, such as three channels, of the colour space isdetermined. The determined single average over a plurality of colourspace channels may then be used as information for random forest as afeature variable.

Average H, S, V values may also be calculated at this stage and thenused in rules-based colour assignment for target variable in trainingdata for the random forest.

Thus, it will be appreciated that the random forest model may learn fromfeatures different from the rules-based approach so as to not simplylearn using a rules-based approach.

This has the advantage that the colour classification algorithm is moreaccurately identify outlier cases such as the example of a black bagwith a red ribbon.

Accordingly, it will be appreciated that with reference to FIG. 6 of thedrawings, two different average colours of selected portions of theimage may be determined.

Firstly, a single average (RGB) value may be determined for a portion ofthe image. This first average value may then be input into the randomforest model. Second and further average values for the portion of theimage may also be determined, for example average (H), average (S), andaverage (V) which are input into the rules based algorithm approach ofFIGS. 7 and 8 of the drawings.

However, it will be appreciated that a single average RGB value and theaverages of any one or more of H, S, V could advantageously be used inthe Random Forest model.

Further, embodiments of the invention may omit the average (H), average(S), and average (V) values from the input to the Random Forest modelbecause some embodiments of the invention already use these average H,S, V values in the rules based colour assessment approach. Omitting thisinput removes the possibility of the Random Forest model learning therules based approach of FIG. 8.

Using H, S, V values rather than R, G, B values for the tree algorithmsuch as that shown in FIG. 8 was found to be more robust compared toknown colour determination algorithms.

A further optional step of applying K-means clustering to the RGB valuesassociated with the portion of the image output from the grab cutprocess may be applied with k=5. Thus, the RGB values for the top 5dominant colours may be determined. Selecting the top 5 dominant coloursis particularly advantageous because test results showed that this wasthe optimum number of dominant colours which allows the algorithm towork most efficiently under test conditions. This is because a lower kmeans that outlier data is not correctly handled, such as a red bag witha black strap. Further a higher value of k does not correctly summarisethe data and furthermore has been found too complicated for a machinelearning algorithm to learn. However it will be appreciated that in somecases less or more than 5 dominant colours may be determined. Forexample, the top 3 or 10 dominant colours may also be determined.

The determined average colour value is then input into a rules-basedcolour assignment algorithm in order to assign a colour or classifiedthe colour of the image according to a predetermined colour. Thefollowing code, as well as the colour tree diagram of FIG. 8 describehow the rule-based colour mapping is performed:

In this code:

-   -   score, label, bounding_box=material_model.image_predict(image)    -   grab_cut_image=grab_cut(image, bounding_box)    -   mean_r, mean_g, mean_b=grab_cut_image.reshape(−1,        3).mean(axis=0)    -   Hsv=Rgb_to_hsv(mean_r, mean_g, mean_b)    -   color_clusters=kmeans(k=5, image)[clusters]    -   features=[color_clusters, [mean_r, mean_g, mean_b], rule_color]

Type features:

-   -   material_pred=material_retinanet.predict(img)    -   type_pred=type_retinanet.predict(img)    -   external_pred=external_retinanet.predict(img)    -   features=[material_pred[labels], type_pred[labels],        external_pred[labels]]

Brightness reduction:

TABLE 10 Code defining the HSV to colour function. In this, the labelsbk, gy, wt, bn, rd, yw, be, gn, bu, pu, rd are the colours black, grey,white, brown, red, yellow, beige, green, blue, purple and redrespectively.   h, s, v = rgb_to_hsv(r, g, b)   v = v − 40   r, g, b =hsv_to_rgb(h, s, v) @staticmethod def hsv_to_color(hsv):  # rules of hsvto get colors from iata chart  # h: 0 − 360  # s: 0 − 100  # v: 0 − 100 hsv = (round(hsv[0] * 360), round(hsv[1]) * 100), round(hsv[2] / (255/ 100)))  if hsv[1] < 15:    if hsv[2] < 25:     final_color = ″bk″   elif hsv[2] < 75:     final_color = ″gy″    else:     final_color =″wt″  elif hsv[1] < 25 and hsv[2] < 16:    final_color = ″bk″  elifhsv[0] >= 0 and hsv[0] <= 10:    if hsv[2] < 40:     final_color = ″bn″   else:     final_color = ″rd″  elif hsv[0] >= 11 and hsv[0] <= 50:   if hsv[1] > 40:     if hsv[2] > 45:      final_color = ″yw″ # orange    else:      final_color = ″bn″    else:     if hsv[2] > 50:     final_color = ″be″     else:      final_color = ″bn″  elifhsv[0] >= 51 and hsv[0] <= 60:    final_color = ″yw″  elif hsv[0] >= 61and hsv[0] <= 149:    final_color = ″gn″  elif hsv[0] >= 150 and hsv[0]<= 250:    final_color = ″bu″  elif hsv[0] >= 251 and hsv[0] <= 320:   final_color = ″pu″  elif hsv[0] >= 321 and hsv[0] <= 360:   final_color = ″rd″ return final_color

As will be appreciated from the colour tree diagram of FIG. 8, theaverage colour is mapped to one of a predetermined colour categorisationdefined by colour space values.

Referring to FIG. 8 of the drawings it will be appreciated thatembodiments of the invention may first examine saturation, S, values,then examine S and V values, and then examine look at H values ifneeded.

Examining the values according to this order is particularly beneficialcompared to examining first hue sample values, then saturation samplevalues and then the sample values.

By way of explanation, the S, Saturation values first indicate how greyan image is or in other words the lack of colour. So this is first usedto filter to colours; black, grey and white if under a certain value.

The, V, Values then indicate brightness, say if black if under a certainvalue.

Finally, the H, Hue, values are then used to determine within whichcolour range or ranges a bag may be in and may again be checked with Vif two colours are close in H values.

This is beneficial because in practice, light sources have a colourtemperature that is not always constant. This means that any black, greyor white a hue value which may not be accurate. Accordingly, hue valuesare not checked until the algorithm has determined with enough certaintythat that the bag is not a black, grey or white bag.

In the specific example of the HSV colour mapping shown in FIG. 8, adetermination is first made as to whether the average S value is lessthan a first threshold=15. If the average S value is not less than 15,then the next step is to determine whether the average V value isgreater than a second threshold=25. If the average value is not greaterthan 25, then the bag is categorised as black. Otherwise if the averagevalue is greater than 25, then a determination is made as to whether theaverage V value is greater than a third threshold=75. If the averagevalue is not greater than the third threshold, then the bag iscategorised as grey, but if the average V value is greater than 75, thenthe bag is classified as white.

It will be appreciated that any predetermined bag colour may be mappedto any one of the colours shown in Table 9 in a similar manner.

Using such rule-based approach to define the target variable embodimentsof the invention provide a systematic approach to defining colours. Thissolves the problem of inconsistent labelling of bag colours. Forexample, one person may label a bag as purple while another may considerthat the same bag is in fact red.

This procedure may correctly categorise a bag colour according to apredetermined colour categorisation. However, a problem with thisapproach is that is uses average values. In the case of a bag having adark strap around it, the average colour is distorted because of thestrap or other dark portion within the bounding box.

Similarly, if a black bag has red strap around it, then the rules basedapproach may categorise the bag as red, because the average colour isused. However, the red colour only occur in low proportion in theportion of the image extracted by the grab cut function.

This problem may be solved by applying a K-means clustering function 617to the RGB values associated with the portion of the image output fromthe grab cut process. K-means clustering is a well-known Open Sourcealgorithm which will be known to the skilled person available athttps://docs.opencv.orq/3.0-beta/doc/py_tutorials/py_ml/py_kmeans/py_kmeans_opencv/py_kmeans_opencv.htmlwhich may be used with k=5, in order to obtain colour values, such asRGB values for the top 5 dominant colours in the portion of the image.

The dominant colour RGB values and the average colour RGB values areused as predictor values to train with using a machine learningalgorithm. Thus, these are the features. The machine learning algorithmmay be advantageously trained using a Random Forest Model 619. This hasthe benefit that it has easily configurable parameters to adjust foroverfitting by tuning hyperparameters.

In other words, training allows embodiments of the invention to learnhow to correctly classify most of the colours using information outsideof a rules-based approach. Any outlier cases to the rules-based approachsuch as a black bag with red strap, are assumed to be a small minorityof data points thus, which have little impact on the model's prediction.This allows for prediction closer to an item's true colour.

It will be appreciated that the random forest model 619 may be themachine learning algorithm used to predict the colour based on theaverage colour values associated with the portion of the image where theitem is located. As shown in FIG. 6 of the drawings, usually the randomforest model 619 receives information defining i) the k dominant coloursand ii) the average colour associated with the portion of the imagewhere the item is located as well as iii) the training data generatedfrom a rules based colour assignment.

However, the random forest model 619 may predict the colour based on anyone or more of the 3 inputs i), ii) or iii) above. This is becauseinputs or techniques i), ii) and iii) are different techniques todescribe colour so theoretically should be complete enough to predictthe colour. Combining these techniques provides an optimum colourdetermination algorithm or in other words provides a good balance forgood indicators of colour and regularisation.

The model predict box shown in FIG. 6 indicates that the final outputpredicted colour output 621 will usually change depending on thetraining data set 615.

Embodiments of the colour categorisation process have the advantage thatit has no human interaction, and avoids inconsistent colourcategorisation.

Accordingly, it will be appreciated that by using a random forests model619 to learn from the dominant colour features allows for correctclassification of problematic images as previously described. Thepreviously determined colour HSV values determined above become targetvalues for machine learning.

Accordingly, it will be appreciated that colour categorisation may beperformed using a rule-based algorithm and with both unsupervised andsupervised machine learning.

The unsupervised approach to create the input data features and arules-algorithm to generate the target variable for inputs into amachine learning algorithm to predict for future bag images.

Thus, a grab-cut function may be performed on all bag images. Thebounding box required by grab-cut may be obtained by first training aRetinaNet model for just the bag bounding box and using the modelbounding box output for the highest score of the bag.

For features embodiments of the invention use k-means clustering withk=5 as described above and use the RGB values for each cluster, we alsotake the average colour by taking the average value of all remaining RGBvalues. Thus, we obtain 3×5+3=18 features.

The target variable is created again by using grab-cut then taking theaverage RGB values and running these values into a manually createdfunction which converts this to HSV and then makes an approximation forthe colour as previously described.

As shown in FIG. 9 the results of the colour classification orcategorisation may be output to a User Interface, at step 709. This maybe advantageously used by an agent for improved retrieval of a lost bag.

Multicolour or Patterned Design Model

A model to detect for if a bag is not a solid colour but rather is amulticolour or patterned design may be provided in some embodiments. Themay be achieved using the previously described bounding box of the bagto train with the labels of material, type and pattern to train forthose models respectively.

The system 100 may interact with other airport systems in order tooutput the determined bag type or/and colour to other systems.

This may be performed by way of Web Services Description Language, WSDL,Simple Object Access Protocol (SOAP), or Extensible Mark-up Language,XML, or using a REST\JSON API call but other messaging protocols forexchanging structured information over a network will be known to theskilled person.

From the foregoing, it will be appreciated that the system, device andmethod may include a computing device, such as a desktop computer, alaptop computer, a tablet computer, a personal digital assistant, amobile telephone, a smartphone. This may be advantageously used tocapture an image of a bag at any location and may be communicativelycoupled to a cloud web service hosting the algorithm.

The device may comprise a computer processor running one or more serverprocesses for communicating with client devices. The server processescomprise computer readable program instructions for carrying out theoperations of the present invention. The computer readable programinstructions may be or source code or object code written in or in anycombination of suitable programming languages including proceduralprogramming languages such as C, object orientated programming languagessuch as C#, C++, Java, scripting languages, assembly languages, machinecode instructions, instruction-set-architecture (ISA) instructions, andstate-setting data.

The wired or wireless communication networks described above may bepublic, private, wired or wireless network. The communications networkmay include one or more of a local area network (LAN), a wide areanetwork (WAN), the Internet, a mobile telephony communication system, ora satellite communication system. The communications network maycomprise any suitable infrastructure, including copper cables, opticalcables or fibres, routers, firewalls, switches, gateway computers andedge servers.

The system described above may comprise a Graphical User Interface.Embodiments of the invention may include an on-screen graphical userinterface. The user interface may be provided, for example, in the formof a widget embedded in a web site, as an application for a device, oron a dedicated landing web page. Computer readable program instructionsfor implementing the graphical user interface may be downloaded to theclient device from a computer readable storage medium via a network, forexample, the Internet, a local area network (LAN), a wide area network(WAN) and/or a wireless network. The instructions may be stored in acomputer readable storage medium within the client device.

As will be appreciated by one of skill in the art, the inventiondescribed herein may be embodied in whole or in part as a method, a dataprocessing system, or a computer program product including computerreadable instructions. Accordingly, the invention may take the form ofan entirely hardware embodiment or an embodiment combining software,hardware and any other suitable approach or apparatus.

The computer readable program instructions may be stored on anon-transitory, tangible computer readable medium. The computer readablestorage medium may include one or more of an electronic storage device,a magnetic storage device, an optical storage device, an electromagneticstorage device, a semiconductor storage device, a portable computerdisk, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), a static random access memory (SRAM), a portable compact discread-only memory (CD-ROM), a digital versatile disk (DVD), a memorystick, a floppy disk.

Exemplary embodiments of the invention may be implemented as a circuitboard which may include a CPU, a bus, RAM, flash memory, one or moreports for operation of connected I/O apparatus such as printers,display, keypads, sensors and cameras, ROM, a communications sub-systemsuch as a modem, and communications media.

The following examples are hereby included to provide furtherdescription of the invention.

1. An item classification system for use in an item handling system, theclassification system comprising:

a. processing means configured to:

i. process an image of an item to determine, based on a first model(101), one or more predetermined first item types, each first item typedefined by one or more first item characteristics;

ii. process the image to determine, based on the first model (101), afirst probability associated with each first item type wherein eachfirst probability is indicative of the likelihood that the item has thefirst characteristics defining each determined first item type;

iii. process the image to determine, based on a second model (103), oneor more predetermined second item types, each second item type definedby one or more second item characteristics;

iv. process the image to determine, based on the second model (103), asecond probability associated with each second item type wherein eachsecond probability is indicative of the likelihood that the item has thesecond characteristics defining each second item type; and

v. classifying the item according to each first item type and eachsecond item type and the probability associated with each first itemtype and the probability associated with each second item type.

2. The system of clause 1 wherein the processing means is furtherconfigured to:

i. process the image to determine, based on a third model (105), one ormore predetermined third item types, each third item type defined by oneor more third item characteristics;

ii. process the image to determine, based on the third model (105), athird probability associated with each third item type wherein eachthird probability is indicative of the likelihood that the item has thethird characteristics defining each third item type; and

iii. further classify the item according to each third item type and theprobability associated with each third item type.

3. The system according to any preceding clause wherein the processingmeans is further configured to;

i. output each determined first item type and the probability associatedwith each determined first item type to a fourth model (107);

ii. output each determined second item type and the probabilityassociated with each determined second item type to the fourth model(107);

iii. preferably output each determined third item type and theprobability associated with each determined third item type to thefourth model (107);

iv. classify, based on the fourth model, the item according to commoncharacteristics of the types determined by the first, second andpreferably the third model.

4. The system according to any preceding clause wherein the processingmeans is further configured to:

i. determine one or more first labels associated with each first itemtype;

ii. determine one or more second labels associated with each second itemtype; and

iii. determine one or more third labels associated with each third itemtype.

5. The system according to any preceding clause wherein the first modelis different from the second model and wherein the second model isdifferent from the third model and wherein the third model is differentfrom the first model.

6. The system according to any preceding clause wherein the itemscomprise baggage.

7. The system according to any preceding clause wherein the itemhandling system is a baggage handling system at an airport.

8. The system according to any preceding clause wherein each first labelor item type is associated with a different type of baggage.

9. The system according to any preceding clause wherein each secondlabel or item type is associated with different external characteristicof baggage.

10. The system according to any preceding clause wherein each thirdlabel or item type is associated with a different characteristicmaterial of baggage.

11. The system of any preceding clause wherein the first model (101) istrained with the or a first label and a plurality of item trainingimages and wherein the second model (103) is trained using the or asecond label and the plurality of item training images and preferablywherein the third model (105) is trained using the or a third label andthe plurality of item training images, and further preferably whereinthe first label is different from the second label and more preferablywherein each of the first label and second label is different from thethird label.

12. The system of any preceding clause wherein the item is classifiedusing a single image.

13. The system of any preceding clause further comprising determining afirst zone from the received image wherein the first zone defines a subregion of the received image which encloses the item and wherein thefirst zone is associated with a first set of sample values.

14. An item handling system comprising the classification system of anypreceding clause wherein the item handling system is arranged to processthe item according to the determined item type.

15. The item handling system comprising the classification system of anypreceding clause wherein the item handling system comprises imagecapture means (109) and/or a conveyor means wherein the item isclassified in response to the image capture means and/or the conveyormeans detecting the presence of the item.

16. The item handling system of clauses 13 or 14 further comprisingsearching a data base or storage means for an item havingcharacteristics corresponding to the determined classification of theitem.

17. The item handling system of clause 15 further comprising searching adatabase for items having associated location data and data defining atime when the item was detected at a particular location.

18. An image processing system for categorising the colour of an item,the system comprising:

processing means configured to:

-   -   i. process an image of an item to extract a portion of the image        where the item is located;    -   ii. determine a plurality of average colour values associated        with the portion of the image wherein the colour values are        associated with a colour space;    -   iii. mapping the plurality of average colour values to one of a        plurality of predetermined colour definitions based on a        plurality of colour ranges associated with each colour        definition.

19. The system of clause 18 wherein the processing means is furtherconfigured to determine a plurality of dominant colours associated withthe portion of the image.

20. The system of clause 18 or 19 wherein the processing means isfurther configured to categorise the colour of the item using a furthermodel 619 based on the determined average colour and the dominantcolours associated with the image.

21. A method for executing the system of any preceding clause.

22. A computer program product which when executed undertakes the methodof clause 21.

23. An image processing system for categorising the colour of an item,the system comprising:

processing means configured to:

-   -   i. process an image of an item to extract a portion of the image        where the item is located;    -   ii. determine a first average colour value of a plurality of        colour values associated with the portion of the image where the        item is located;    -   iii. mapping the average colour value to one of a plurality of        predetermined colour definitions based on a plurality of colour        ranges associated with each colour definition; and    -   iv. categorising the colour of the item according to the        mapping.

24. The system of clause 23 wherein the first average colour value isdetermined according to a first colour space and wherein the processingmeans is further configured to determine a second average colour valueaccording to a second colour space wherein the first colour space isdifferent from the second colour space.

25. The system of clause 24 wherein the processing means is furtherconfigured to determine a single average colour for the plurality ofchannels defining the first colour space and preferably to determine aplurality of average colours for each of the plurality of channelsdefining the second colour space.

26. The system of clause 24 or 25 wherein the first colour space is ared, green, blue colour space and preferably wherein the second colourspace is a hue, saturation and value colour space.

27. The system of any preceding clause wherein the processing means isfurther configured to determine a plurality of dominant coloursassociated with the portion of the image and preferably to determinecolour values for each of the plurality of dominant colours and furtherpreferably wherein each dominant colour is determined using k-meansclustering.

28. The system of any preceding clause wherein the processing means isfurther configured to categorise the colour of the item using a model(619) based on the determined average colour value or/and the dominantcolours associated with the image.

29. The system of claim 28 further comprising training the model (619)using the determined dominant colour values or/and the average colourvalues and preferably wherein the model is a random forest model (619).

30. The system of any preceding clause wherein the processing means isfurther configured to generate training data using the image of the itemwherein the training data comprises any one or more of:

-   -   a. the or an image of the item;    -   b. the or an associated colour classification of the item;    -   c. the or an average colour values of the item; and    -   d. the or further dominant colour values of the item.

31. The system of clause 30 wherein the average colour values and thedominant colour values are determined from an extracted portion of theimage where the item is located.

32. A baggage handling system comprising the image processing system ofclause 23 wherein the item is an item of baggage for check in at anairport.

33. The baggage handling system of any preceding clause wherein thesystem is configured to categorise the colour of the item in response toa passenger or agent placing the item of baggage on a bag drop belt.

34. The baggage handling system of any preceding clause wherein theprocessing means is further configured to determine a bounding boxenclosing at least a portion of the bag within the image.

35. The baggage handling system of any preceding clause wherein theprocessing means is further configured to extract a portion of the imagecontaining the bag using a grab cut function.

36. An image processing system for categorising the colour of an item,the system comprising:

processing means configured to:

-   -   i. process an image of an item to extract a portion of the image        where the item is located;    -   ii. determine a first average colour value of a plurality of        colour values associated with the portion of the image where the        item is located;    -   iii. determine a plurality of dominant colour values associated        with the portion of the image where the item is located; and    -   iv. categorising the colour of the item according to the first        average colour value and the plurality of dominant colour        values.

37. A method for executing the system of any preceding clause andpreferably a computer program product which when executed undertakes themethod.

1. An item classification system for use in an item handling system, theclassification system comprising: a. processing means configured to: i.process an image of an item to determine, based on a first model (101),one or more predetermined first item types, each first item type definedby one or more first item characteristics; ii. process the image todetermine, based on the first model (101), a first probabilityassociated with each first item type wherein each first probability isindicative of the likelihood that the item has the first characteristicsdefining each determined first item type; iii. process the image todetermine, based on a second model (103), one or more predeterminedsecond item types, each second item type defined by one or more seconditem characteristics; and iv. process the image to determine, based onthe second model (103), a second probability associated with each seconditem type wherein each second probability is indicative of thelikelihood that the item has the second characteristics defining eachsecond item type v. classify the item according to each first item typeand each second item type and the probability associated with each firstitem type and the probability associated with each second item type. 2.The system of claim 1 wherein the processing means is further configuredto: i. process the image to determine, based on a third model (105), oneor more predetermined third item types, each third item type defined byone or more third item characteristics; ii. process the image todetermine, based on the third model (105), a third probabilityassociated with each third item type wherein each third probability isindicative of the likelihood that the item has the third characteristicsdefining each third item type; and iii. further classify the itemaccording to each third item type and the probability associated witheach third item type.
 3. The system according to any preceding claimwherein the processing means is further configured to: i. output eachdetermined first item type and the probability associated with eachdetermined first item type to a fourth model (107); ii. output eachdetermined second item type and the probability associated with eachdetermined second item type to the fourth model (107); iii. preferablyoutput each determined third item type and the probability associatedwith each determined third item type to the fourth model (107); iv.classify, based on the fourth model, the item according to commoncharacteristics of the types determined by the first, second andpreferably the third model.
 4. The system according to any precedingclaim wherein the processing means is further configured to: i.determine one or more first labels associated with each first item type;ii. determine one or more second labels associated with each second itemtype; and iii. determine one or more third labels associated with eachthird item type.
 5. The system according to any preceding claim whereinthe first model is different from the second model and wherein thesecond model is different from the third model and wherein the thirdmodel is different from the first model.
 6. The system according to anypreceding claim wherein the items comprise baggage.
 7. The systemaccording to any preceding claim wherein the item handling system is abaggage handling system at an airport.
 8. The system according to anypreceding claim wherein each first label or item type is associated witha different type of baggage.
 9. The system according to any precedingclaim wherein each second label or item type is associated withdifferent external characteristic of baggage.
 10. The system accordingto any preceding claim wherein each third label or item type isassociated with a different characteristic material of baggage.
 11. Thesystem of any preceding claim wherein the first model (101) is trainedwith the or a first label and a plurality of item training images andwherein the second model (103) is trained using the or a second labeland the plurality of item training images and preferably wherein thethird model (105) is trained using the or a third label and theplurality of item training images, and further preferably wherein thefirst label is different from the second label and more preferablywherein each of the first label and second label is different from thethird label.
 12. The system of any preceding claim wherein the item isclassified using a single image.
 13. The system of any preceding claimfurther comprising determining a first zone from the received imagewherein the first zone defines a sub region of the received image whichencloses the item and wherein the first zone is associated with a firstset of sample values.
 14. An item handling system for use in the airtransportation industry comprising the classification system of anypreceding claim wherein the item handling system is arranged to processthe item according to the determined item type.
 15. The item handlingsystem according to claim 14 comprising a plurality of different itemprocessing channels comprising first and second item processing channelsand preferably wherein the item handling system is configured to selectthe first or second item processing channel depending upon thedetermined item type and further preferably to divert the item along thefirst or second item processing channel depending upon the determineditem type.
 16. The item handling system comprising the classificationsystem of any preceding claim wherein the item handling system comprisesimage capture means (109) and/or a conveyor means wherein the item isclassified in response to the image capture means and/or the conveyormeans detecting the presence of the item.
 17. The item handling systemof claims 14 to 16 further comprising searching a database or storagemeans for an item having characteristics corresponding to the determinedclassification of the item.
 18. The item handling system of claims 14 to17 further comprising searching a database for items having associatedlocation data and data defining a time when the item was detected at aparticular location.
 19. The item classification system or the itemhandling system of any preceding claim wherein the processing means isfurther configured to: i. process the image of an item to extract aportion of the image where the item is located; ii. determine a firstaverage colour value of a plurality of colour values associated with theportion of the image where the item is located; iii. mapping the averagecolour value to one of a plurality of predetermined colour definitionsbased on a plurality of colour ranges associated with each colourdefinition; and iv. categorising the colour of the item according to themapping.
 20. The system of claim 19 wherein the first average colourvalue is determined according to a first colour space and wherein theprocessing means is further configured to determine a second averagecolour value according to a second colour space wherein the first colourspace is different from the second colour space.
 21. The system of claim20 wherein the processing means is further configured to determine asingle average colour for the plurality of channels defining the secondcolour space and preferably to determine a plurality of average coloursfor each of the plurality of channels defining the first colour space.22. The system of claim 20 or 21 wherein the second colour space is ared, green, blue colour space and preferably wherein the first colourspace is a hue, saturation and value colour space.
 23. The system of anyone of claims 19 to 22 wherein the processing means is furtherconfigured to determine a plurality of dominant colours associated withthe portion of the image and preferably to determine colour values foreach of the plurality of dominant colours and further preferably whereineach dominant colour is determined using k-means clustering.
 24. Thesystem of any one of claims 19 to 23 wherein the processing means isfurther configured to categorise the colour of the item using a model(619) based on the determined average colour value or/and the dominantcolours associated with the image.
 25. The system of claim 24 furthercomprising training the model (619) using the determined dominant colourvalues or/and the average colour values and preferably wherein the modelis a random forest model (619).
 26. The system of any one of claims 19to 25 wherein the processing means is further configured to generatetraining data using the image of the item wherein the training datacomprises any one or more of: a. the or an image of the item; b. the oran associated colour classification of the item; c. the or an averagecolour values of the item; and d. the or further dominant colour valuesof the item.
 27. The system of claim 26 wherein the average colourvalues and the dominant colour values are determined from an extractedportion of the image where the item is located.
 28. The system of anypreceding claim wherein the processing means is further configured todetermine a bounding box enclosing at least a portion of the item withinthe image using the first model (103) or the second model (105) and tooutput data defining the bounding box to a grab cut function configuredto perform foreground or background selection.
 29. The system of anypreceding claim wherein the processing means is further configured toextract a portion of the image containing the item using the or a grabcut function.
 30. An item classification method for use in an itemhandling system, the classification method comprising the steps of:processing an image of an item to determine, based on a first model(101), one or more predetermined first item types, each first item typedefined by one or more first item characteristics; processing the imageto determine, based on the first model (101), a first probabilityassociated with each first item type wherein each first probability isindicative of the likelihood that the item has the first characteristicsdefining each determined first item type; processing the image todetermine, based on a second model (103), one or more predeterminedsecond item types, each second item type defined by one or more seconditem characteristics; processing the image to determine, based on thesecond model (103), a second probability associated with each seconditem type wherein each second probability is indicative of thelikelihood that the item has the second characteristics defining eachsecond item type; and classifying the item according to each first itemtype and each second item type and the probability associated with eachfirst item type and the probability associated with each second itemtype.
 31. A computer program product which when executed undertakes themethod of claim 30.